{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 线性回归应用实例——boston房价预测\n",
    "\n",
    "#### 机器学习应用步骤：1. 收集数据，处理数据（清洗，特征提取）——这里我们直接使用sklearn自带数据集，该数据集是美国波士顿地区一批房价信息，影响房价的因素总共定义了13个\n",
    "\n",
    "#### 2. 划分训练集和测试集，使用sklearn的train_test_split 方法\n",
    "\n",
    "#### 3. 选择一个机器学习算法——线性回归 LinearRegression\n",
    "\n",
    "#### 4. 在训练集上训练模型  fit\n",
    "\n",
    "#### 5. 在测试集上测试模型，看模型得分 score\n",
    "\n",
    "#### 6. 模型的改进——发现模型欠拟合，所以增加多项式特征"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "from sklearn.datasets import load_boston\n",
    "\n",
    "boston=load_boston()  #记载数据集\n",
    "X=boston.data  #输入数据，特征数据\n",
    "Y=boston.target   #房价\n",
    "X.shape  #数据集有506个样本，每个样本有13个特征，整个样本放在一个506*13的矩阵中"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "X[0]  #查看第一个样本数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Y[0]  #查看第一个样本的标签，即房价"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "boston.feature_names  #查看特征的标签"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#训练集：测试集=8:2或者7:3  划分比例\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.linear_model import LinearRegression\n",
    "\n",
    "X_train,X_test,Y_train,Y_test=train_test_split(X,Y,test_size=0.2,random_state=3) #把样本集分割成训练集和测试集\n",
    "# test_size指定了测试集占的比例，random_state是随机数的种子，设定了之后，每次运行划分结果是一样的\n",
    "\n",
    "model=LinearRegression(normalize=True)  #定义一个线性回归模型，normalize=True表示对输入数据做归一化处理，提高算法效率\n",
    "\n",
    "model.fit(X_train,Y_train)  #在训练集上训练模型\n",
    "\n",
    "train_score=model.score(X_train,Y_train)\n",
    "test_score=model.score(X_test,Y_test)\n",
    "\n",
    "\n",
    "print(train_score)  #0.7239410298290111   欠拟合\n",
    "print(test_score)  #0.7949575364750558\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#上述模型出现了欠拟合，所以增加多项式特征\n",
    "\n",
    "from sklearn.preprocessing import PolynomialFeatures\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.externals import joblib\n",
    "\n",
    "def polynomial_model(degree=1):\n",
    "    polynomial_features=PolynomialFeatures(degree=degree,include_bias=False)  #多项式特征\n",
    "    linear_regression=LinearRegression(normalize=True) #线性回归的模型 normalize=True表示对输入数据做归一化处理\n",
    "    pipeline=Pipeline([('polynomial_features',polynomial_features),('linear_regression',linear_regression)])\n",
    "    return pipeline\n",
    "\n",
    "model=polynomial_model(degree=2)  #二阶多项式\n",
    "model.fit(X_train,Y_train)  #在训练集上训练模型\n",
    "\n",
    "train_score=model.score(X_train,Y_train)\n",
    "test_score=model.score(X_test,Y_test)\n",
    "\n",
    "joblib.dump('LRModel2.pkl')  #保存模型\n",
    "\n",
    "\n",
    "print(train_score)  #0.7239410298290111   欠拟合\n",
    "print(test_score)  #0.7949575364750558"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model=polynomial_model(degree=3)   #三阶多项式\n",
    "model.fit(X_train,Y_train)  #在训练集上训练模型\n",
    "\n",
    "train_score=model.score(X_train,Y_train)\n",
    "test_score=model.score(X_test,Y_test)\n",
    "\n",
    "\n",
    "print(train_score)  #1.0   过拟合，所以多项式的阶数并不是越多越好\n",
    "print(test_score)  #-105.51701646615065"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 过拟合，模型能非常好的拟合训练集中的数据，但是对于“没见过的数据”预测能力差（模型 泛化能力差）\n",
    "\n",
    "#### 处理方法：  1. 获取更多的训练数据，2. 减少输入的特征数量\n",
    "\n",
    "#### 对于boston这个列子，想要继续优化，就需要去收集更多的数据集。还有一个办法是换一个算法\n",
    "\n",
    "#### 后面学习了其他算法之后，可以在应用其他算法来对这个问题建模"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
